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SECTION  I 


INTRODUCTION 

The  Microwave  Landing  System  (MLS)  is  an  instrument  landing  system  currently 
under  development.  The  MLS  ground  station  electronically  defines  each  waypoint 
in  the  terminal  approach  airspace  out  to  20NM  in  range  to  the  ground  station, 
up  to  20°  in  elevation  angle,  and  out  to  60°  on  either  side  of  the  runway 
centerline.  Given  such  information,  appropriate  data  processing,  and  the 
application  of  modern  control  techniques, curved  and  segmented  approach  paths 
can  be  defined  and  flown.  Curved  and  segmented  approaches  are  very  different 
from  the  current  ILS  approach.  An  ILS  approach  requires  the  aircraft  to  track  only 
on  the  extended  runway  centerline  and  on  one  glide  path  through  the  terminal  area 
to  landing.  However,  an  MLS  curved  and  segmented  approach  contains  a number  of 
different  courses  and  glide  paths,  all  of  which  interconnect.  ILS  avionics  and 
data  processing,  however,  cannot  generate  usable  course  and/or  glide  slope 
information  through  an  MLS  approach. 

A recent  aircraft  simulation  study,  "MLS  Utilization  with  Conventional 
Avionics,"  examined  how  employing  typical  ILS  data  processing  and  display 
systems  affected  pilot  performance  on  MLS  approaches.  The  study  confirmed  that 
the  systems  were  inadequate  on  crossleg  segments  due  to  a lack  of  course 
guidance.  On  all  profiles,  excessive  course  tracking  errors  developed  on  the 
unguided  segments.  Pilots  found  that  the  navigation  problems  they  had  encoutered 
on  the  unguided  segments  also  adversely  affected  their  final  approach  performances. 
A major  conclusion  from  the  study  was  that  course  and  glide  slope  guidance  should 
be  presented  throughout  the  approach  to  achieve  acceptable  levels  of  performance 
(ASD  TR-76-7). 
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The  study  described  in  this  report  examines  pilot  performance  using  a 
newly  designed  Digital  Flight  Director  System  (DFDS)  which  was  installed  in  a 
T-40  simulator.  The  DFDS  uses  MLS  azimuth,  range,  and  elevation  information 
to  compute  course  and  glide  path  guidance  throughout  MLS  approaches.  DFDS 
operation  requires  the  pilot  to  define  and  input  the  desired  approach  path  as  a 
sequence  of  waypoints  defined  in  terms  of  MLS  azimuth,  range,  and  elevation. 

The  DFDS  computes  a path  in  space  between  each  succeeding  pair  of  waypoints  in 
the  sequence.  During  the  approach,  the  DFDS  continually  calculates  the 
distance  between  the  aircraft  and  the  computed  approach  path  and  presents 
appropriate  course  and  glide  path  guidance  to  the  pilot. 

The  pilot  can  program  the  DFDS  in  flight,  which  allows  him  to  define  each 
approach  individually.  Thus,  an  air  traffic  controller  could  allot  a different 
approach  profile  to  each  aircraft  under  his  control.  Theoretically,  by  the 
use  of  divergent  and  separate  approach  profiles  the  controller  could  safely 
increase  air  traffic  density  in  the  terminal  area,  which  would  use  airspace 
and  runways  more  efficiently.  Further,  properly  developed  approach  profiles 
could  avoid  population  centers  and  create  effective  noise  abatement  procedures. 
The  high  energy  MLS  signal  is  virtually  distortion  free  and  offers  better 
guidance  close  to  touchdown  (TD)  for  Category  II  and  III  approaches.  The  above 
capabilities  make  MLS  utilization  attractive. 

However,  the  operational  flexibility  of  the  MLS  can  cause  some  problems. 

To  safely  fly  MLS  approaches,  especially  in  the  crowded  terminal  area  airspace, 
requires  that  each  aircraft  be  in  its  designated  airspace  at  all  times.  Safe 
MLS  approach  navigation  is  dependent  not  only  upon  fault-free  instrument 
operation  and  accurate  pilot  tracking,  but  also  upon  correct  approach  path 
definition.  The  inflight  pilot  programing  capability  permits  input  errors  which 
could  define  unintended  and  unsafe  nominal  approach  path.  For  example,  a 


pilot  could  intend  to  fly  an  approach  intercepting  the  final  approach  course 
3NM  from  TD,  but  actually  program  an  approach  with  a 1.5NM  intercept  distance 
The  pilot,  when  flying  the  approach,  could  only  detect  such  an  error  using  his 
conventional  displays,  which  were  designed  for  the  geometrical ly  much  simpler 
ILS  approach.  If  the  pilot  failed  to  detect  the  error,  in  all  likelihood  he 
would  fly  out  of  his  designated  airspace  and  into  a potentially  dangerous 
situation. 


This  study  had  two  major  objectives.  The  study  was  designed  to  evaluate 
how  well  a pilot  could  determine  if  he  were  flying  the  intended  approach  path 
and  to  examine  how  profile  design  factors  affected  tracking  performance. 


SECTION  II 
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STUDY  PROCEDURE 

I . APPARATUS 

The  experiment  was  conducted  in  the  Crew  Station  Design  Facility  (CSDF), 
which  has  the  capability  to  dynamically  simulate  a complete  flight  regime 
under  a variety  of  controlled  conditions.  The  experimental  apparatus  was  the 
T-40  simulator,  which  simulates  flight  in  a light  two-engine  jet  aircraft. 

The  T-40  crew  station  shell  was  mounted  on  a motion  platform  with  ^hree  degrees 
of  freedom:  pitch,  15  degrees  down  and  25  degrees  up;  roll,  9 degrees  depending 

on  pitch  angle;  and  + 12  inches  of  vertical  displacement.  The  Mark  I digital 
computer  controlled  all  the  simulator's  functions.  With  the  magnetic  tape 
unit,  data  on  20  separate  parameters  were  recorded  at  intervals  of  0.2  seconds. 
The  parameters  for  this  study  are  in  Appendix  A.  The  visual  simulation  equipment 
was  not  used. 

The  T-40  crew  station  was  configured  as  a T-39  test  aircraft  with  T-39 
aerodynamics  and  instruments  (Figure  1).  The  system  contained  an  Attitude 
Director  Indicator  (ADI),  powered  by  the  newly  designed  Digital  Flight  Director 
System,  with  the  pitch  and  bank  steering  bars  operating  as  processed  (command) 
information.  A glide  slope  indicator  (GS),  located  on  the  left  side  of  the  ADI, 
displayed  unprocessed  (raw)  glide  slope  deviation  via  a pointer  moving  over  a 
vertical  scale.  A rate-of-turn  indicator  and  an  inclinometer  were  located  on  the 
bottom  of  the  ADI.  Located  below  the  ADI  was  the  Horizontal  Situation  Indicator 
(HSI),  which  contained  the  Course  Deviation  Indicator  (CDI).  The  CDI  presented 
unprocessed  (raw)  localizer  information.  A heading  set  knob  and  a course  set 
knob  rotated  a small  white  heading  indicator  about  the  compass.  The  course  set 
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knob  operated  the  digital  course  indicator  in  the  upper  right  portion  of  the 
HSI  and  could  reposition  the  CDI  when  the  aircraft  was  receiving  course 
information.  The  distance  from  the  MLS  azimuth  transmitter  was  given  by  the 
three-digit  DME  readout  in  the  upper  left  corner. 

Mounted  between  the  ADI  and  the  HSI  was  the  Azimuth  Angle  Indicator  (AAI) 
(Figure  2).  It  displayed  aircraft  position  (pointer)  in  relation  to  a scale 
marked  in  degrees  representing  MLS  radial s.  The  0°  mark  referred  to  the 
extended  runway  centerline  and  was  located  at  the  center  of  the  scale.  The 
scale  extended  out  to  60°  either  side  of  the  0°  mark  in  a nonlinear  fashion, 
with  the  larger  units  nearer  the  0°  mark.  The  indication  displayed  in  Figure  2 
marks  aircraft  position  at  some  point  on  the  20°  left  radial,  which  corresponds 
to  the  radial  used  for  the  initial  leg  of  profiles,  1,  2,  and  4. 

The  simulator  instrument  system  included  the  DFDS,  which  replaced  the 
standard  ILS  analog  flight  director  system.  The  DFDS  did  not  physically  change 
the  guidance  displays,  but  did  change  their  information  sources.  In  the  DFDS 
range  to  the  transmitter  (DME),  bearing  to  the  station,  HSI  course  readout,  and 
CDI  orientation  are  all  dependent  upon  and  updated  by  the  DFDS.  However,  the 
most  dramatic  change  from  the  ILS  concerns  the  relationship  of  processed  and 
unprocessed  information. 

An  ILS  airborne  receiver  is  designed  to  detect  aircraft  position  about 
the  ILS  beam  which,  due  to  the  geometric  simplicity  of  the  ILS  approach,  is 
actually  course  and  glide  slope  error.  The  ILS  receiver  sends  its  outputs  to 
the  CDI  and  G/S  indicator  where  they  are  displayed  as  course  error  and  glide 
slope  error,  respectively.  These  same  ILS  receiver  outputs  are  also  sent  to 


the  analog  flight  director  system,  which  further  processes  the  information  and 
presents  it  as  command  guidance  on  the  pitch  and  bank  steering  bars.  In  the 


Figure  2.-  Azimuth  Angle  Indicator 


1 


ILS  system,  the  unprocessed  information  displays  give  the  pilot  a form  of 
position  information  which  can  be  used  to  determine  aircraft  location  in  the 
terminal  area. 

The  DFDS  is  much  different.  The  broad  MLS  coverage,  more  complex  MLS 
approach  design,  and  the  variety  of  unique  MLS  approaches  available  have 
necessitated  that  the  DFDS  compute  processed  and  unprocessed  information  and 
send  such  information  to  the  appropriate  displays.  The  MLS  receiver  receives 
aircraft  position  information,  which  is  defined  in  terms  of  azimuth  angle, 
elevation  angle,  and  range.  The  position  information  is  sent  to  the  DFDS 
where  the  distance  between  the  aircraft  and  the  nominal  track  is  computed. 

Pn  this  basis,  course  and  glide  slope  error  values  are  generated.  At  this  point, 
course  error  and  glide  slope  error  are  sent  to  and  displayed  by  the  CDI  and 
G/S  indicator,  respectively.  However,  the  course  and  glide  slope  error  values 
a-e  also  sent  to  other  portions  of  the  DFDS  where  they  are  further  processed 
to  generate  command  guidance  signals  which  are  presented  by  the  pitch  and  bank 
steering  bars.  Both  processed  and  unprocessed  guidance,  as  in  the  ILS,  refer 
to  aircraft  position  relative  to  the  nominal  track.  However,  since  in  the  MLS 
environment  a pilot  could  be  flying  cn  one  of  many  different  approach  paths, 
he  must  crosscheck  his  absolute  location  at  all  times.  To  do  this,  he  must 
use  and  integrate  other  sources  of  information  such  as  DME,  bearing,  and  altitude. 

The  pilot's  access  to  the  DFDS  is  through  the  Navigation  Control  Display 
Unit  (NCDU)  located  in  the  center  of  the  instrument  console.  A pilot  could 


A mode  progress  display,  located  left  of  the  ADI,  was  a three-window 
display  used  to  aid  the  pilot  during  capture  and  throughout  the  approach. 

Glide  slope  (G/S)  and  localizer  (LOC)  capture  were  indicated  by  the  mode 
progress  display.  When  capture  occurred,  the  appropriate  display  changed  from 
green  to  orange.  The  mode  progress  display  also  notified  the  pilot  when  the 
final  approach  leg  was  encountered  by  displaying  an  orange  "CAT  II"  symbol. 

2.  SUBJECTS 

Fourteen  USAF  pilots  from  the  4950th  Test  Wing  participated  as  subject 
pilots.  The  group  averaged  29  years  of  age,  6.4  years  service  time,  and 
1641  hours  of  flight  time. 

3.  EXPERIMENTAL  DESIGN 

The  experiment  was  a completely  randomized  block  design  which  employed  two 
factors.  The  first  factor  was  the  type  of  unintended  flight  path.  The  second 
factor,  profile  design,  dealt  with  how  various  aspects  of  profile  design 
affected  tracking  performance. 

During  testing,  every  subject  flew  a random  sequence  of  40  test  approaches. 
The  sequence  differed  for  each  subject,  but  every  subject  flew  each  of  the 
eight  profiles  (Figure  3)  five  times  (see  Appendix  C for  details).  Superimposed 
over  every  40  trial  sequence  was  a random  sequence  of  ten  discrepant  (see 
Subsection  1 1 1-4  Discrepancy  Design)  and  30  nondiscrepant  conditions.  Each 
such  sequence  was  varied  from  subject  to  subject,  but  always  included  two 
instances  of  discrepancies  types  I,  IIA,  and  I IB  and  one  occurrence  of  types 
1 1 1 AX , IIIAY,  IIIBX,  and  1 1 1 BY . 

During  a discrepant  condition  the  experimenter,  unbeknown  to  the  subject, 
altered  the  DFDS  software  to  introduce  one  of  the  seven  navigation  discrepancies. 
On  nondiscrepant  trials,  the  pilot  flew  the  intended  profile. 


4.  DISCREPANCY  DESIGN 


Presented  in  Figures  4,  5,  and  6 are  the  navigation  discrepancies  used  in 
this  experiment.  Listed  in  Table  1 are  brief  descriptions  of  each  discrepancy 
and  a shcrt  explanation  of  how  each  one  could  be  detected.  Additional 
information  on  the  pilot  procedures  required  to  detect  each  discrepancy  are 
giver,  i n Appendix  B . 

The  discrepancies  v;ere  designed  to  highlight  navigation  problems  that 
could  create  unsafe  conditions,  especially  in  Category  III--weather.  The 
nature  of  the  present  DFDS  design  affected  discrepancy  implementation.  Some 
could  be  implemented  by  simulating  one  simple  pilot  programming  error  while 
others  required  a more  complex  simulation.  In  either  case,  the  effort  was  to 
create  problematic  navigation  conditions  and  avoid  devising  discrepancies 
unique  to  the  present  DFDS  design. 

5.  PROCEDURE 

The  experimenter  briefed  every  subject  on  the  purpose  of  the  experiment 
and  about  the  MLS  approaches  and  the  DFDS.  He  also  described  in  general  terms 
what  a navigation  discrepancy  was  and  cited  each  type  of  guidance  and  position 
information  and  traced  its  source.  Although  the  pilots  were  not  given  exact 
descriptions  of  the  discrepancies  they  were  to  receive,  they  were  told  to 
monitor  course,  heading,  turn  point  range,  altitude,  and  rate  descent  and 
given  detection  criteria.  4fterward,  the  experimenter  escorted  the  subjects 
to  the  simulator  and  showed  them  how  to  operate  the  DFDS. 

After  the  briefing  and  DFDS  demonstration,  each  subject  flew  two  sets  of 
five  practice  approaches  in  which  all  eight  profiles  were  flown  once,  except 
for  profiles  5 and  6,  which  were  flown  twice.  No  discrepancies  were  introduced 
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*See  Appendix  B for  more  detail. 


during  practice.  During  testing,  every  subject  flew  two,  five  approach  sets 
daily  and  rested  between  each  set.  After  a set,  each  pilot  answered  the  MLS 
Post  Mission  Questionnaire  (see  Appendix  D).  After  completing  the  40  test 
approaches,  each  answered  the  MLS  end  of  Study  Questionnaire  (see  Appendix  D). 

To  start  an  approach,  the  pilot  input  the  intended  waypoint  sequence  into 
the  DFDS  while  the  experimenter  set  the  aircraft  simulator  at  the  proper 
altitude,  location,  heading,  and  airspeed.  Upon  the  pilot's  request,  the 
experimenter  transferred  control  of  the  aircraft  simulator  from  the  computer 
to  the  pilot.  After  transfer  of  control,  the  pilot  could  use  only  the  CDI 
and  GS  guidance  displays,  since  the  DFDS  did  not  present  command  guidance  until 
the  pilot  intercepted  the  nominal  path.  Once  intercepted,  the  steering  bars 
displayed  command  guidance  and  did  so  throughout  the  approach. 


6.  TRACKING  PERFORMANCE  SCORING  SYSTEM 

Each  MLS  pattern  was  divided  into  five  zones  for  scoring.  Zone  5,  the 
final  approach,  terminated  at  200  ft  above  field  elevation.  Zone  4,  the 
turn  onto  the  final  approach,  connected  zone  5 with  zone  3,  the  crossleg. 

Zone  2,  a turn,  connected  zone  3 with  zone  1,  an  initial  portion  of  each 
approach  profile. 

Within  each  leg,  four  parameters  were  tabulated  to  describe  the  aircraft's 
performance.  Airspeed  error  score  (A/S),  lateral  tracking  error  score  (Y), 
vertical  tracking  error  score  (Z),  and  a combination  score  (YZ)  consisting  of 
both  lateral  and  vertical  tracking  error,  were  collected.  Each  Y and  Z value 
represents  the  actual  distance  in  feet  of  the  aircraft  from  the  nominal  track, 
laterally  and  vertically.  Each  YZ  value  is  the  distance  in  feet  between  the 
aircraft  and  the  desired  track  and  was  calculated  using  the  Y and  Z value  at 
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each  interaction  and  the  Pythagorean  Theorerr.  Each  airspeed  value  was  the 
difference  between  the  aircraft  indicated  airspeed  and  140  kts  at  each  iteration. 

I Each  A/S,  Y,  Z and  YZ  value  was  computed  every  0.2  second,  an  absolute  value 

determined,  summed  with  the  other  values  of  that  parameter  for  that  leg,  and 
divided  by  the  number  of  iterations  during  that  leg  to  produce  a score  for  that 
parameter  on  that  leg.  Each  parameter  value  and  score  (absolute  average  error 
term)  was  calculated,  depending  upon  leg  number  and  pattern  number. 

Essentially,  the  scoring  system  tabulated  a set  of  absolute  average  error 
(AAF)  terms  which  described  how  closely  the  pilot  flew  the  aircraft  to  the 
nominal  track  at  the  desired  airspeed  for  each  leg.  Each  AAE  term  reflects 
not  only  consistently  poor  performance,  but  also  tracking  performance  which 
oscillates  about  the  nominal  track.  Differences  among  AAE  terms  for  a 
particular  parameter  accurately  reflect  differences  in  tracking  performar.ee. 

The  AAE  terms,  by  their  nature,  are  normalized  for  time  and  distance  and 
therefore  permit  comparisons  across  pattern  types  (see  ASD  TR-76-7  for  more 
detail ). 


SECTION  III 


RESULTS 

The  data  were  divided  into  two  groups.  On  trials  which  did  not  involve 
a waypoint  discrepancy,  tracking  performance  scores  were  obtained.  On  approaches 
involving  a waypoint  discrepancy,  the  data  of  interest  were  the  pilot's  abilities 
to  detect  the  discrepancies. 

The  tracking  performance  scores  for  the  crossleg  are  graphically  depicted 
in  Figure  7.  The  crossleg  is  the  straight  leg  segment  immediately  preceding 
the  final  approach.  For  each  profile,  the  mean  lateral  error  and  the  mean 
vertical  error  scores  are  displayed,  which  make  up,  as  discussed  in  the  Scoring 
System  section,  the  mean  combination  score.  Figure  7 shows  that  on  each  profile 
the  mean  lateral  error  score  greatly  exceeded  the  mean  vertical  error  score. 

In  fact,  the  mean  lateral  error  score  was  almost  as  large  as  the  mean  combination 
score,  which  left  the  mean  vertical  error  scores  relatively  insignificant.  This 
finding  demonstrates  that  controlling  lateral  error  was  the  predominant  problem 
on  the  crossleg. 

An  analysis  of  variance  was  performed  on  the  crossleg  mean  combination 
scores  across  profiles.  The  results  (Table  2)  show  that  the  type  of  profile 
flown  significantly  affected  performance  (p  <_  .01).  To  evaluate  which 
differences  among  the  profile  mean  scores  were  significant,  a Newman-Kuel s 
test  was  performed.  The  Newman-Keuls  test,  designed  to  perform  pair-wise 
comparisons,  revealed  that  the  exceptionally  high  mean  combination  scores  for 
profiles  2 and  4 differed  significantly  (p  <_  .01)  from  the  other  profiles. 

The  comparisons  also  demonstrated  that  the  mean  combination  scores  for  profiles 
2 and  4 demonstrated  the  relative  performance  difficulty  of  these  profiles. 

Final  approach  performance,  graphically  pictured  in  Figure  8,  depicts 
tracking  performance  across  profiles.  On  each  profile  the  mean  combination 
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TABLE  2 


ANALYSIS  OF  VARIANCE  SUMMARY  TABLE 
CROSSLEC  TRACKING  PERFORMANCES 


Source 

Sum  of  Squares 

df 

Mean  Square 

' K.it  io 

Profile  Type  (A) 

685,633.3 

7 

97,947.6 

**  11.95 

Subject  (S) 

5,338,466.5 

13 

410,651.3 

Error  Term 

745,721.7 

91 

8,194.7 

(AS) 


**  11.95;  p .01  F (7,91) 
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FINAL  APPROACH  PERFORMANCE 


■ 


error  score  is  separated  into  its  mean  lateral  error  and  mean  vertical  error 
scores.  A second  analysis  of  variance  was  performed  on  the  final  approach 
mean  combination  error  scores  across  profiles  and  the  results  are  displayed 
in  Table  3.  The  analysis  revealed  that  the  type  of  profile  flown  significantly 
affected  performance  (p  <_  .01).  A Newman-Keuls  test  was  performed  and  the  test 
results  are  depicted  in  Table  4.  Contained  in  Appendix  E is  a breakdown  of 
crossleg  and  final  approach  performance  showing  lateral  and  vertical  error 
scores  per  profile  as  well  as  the  combination  score. 

The  data  from  trials  with  discrepant  waypoints  is  displayed  as  percent 
detection  in  Table  5.  A Friedman  Two-Way  analysis  of  variance  was  done  on 
the  data  and  showed  that  the  percent  detection  was  a function  of  the  type  of 
discrepancy  given.  The  analysis  showed  a marked  difference  in  the  percent 
detection  between  lateral  situation  discrepancies  (I,  IIA,  I IB ) and  vertical 
situation  discrepancies  (IIIAX,  IIIAY,  IIIBX,  1 1 1 BY ) , revealing  that  lateral 
types  were  more  detectable.  This  not  to  convey  that  only  vertical  situation 
discrepancies  were  problematic.  On  average,  the  lateral  situation  discrepancies 
were  noted  only  78  percent  of  the  time,  which  still  left  22  percent  of  the 
errors  undetected.  With  about  a quarter  of  the  lateral  errors  unnoticed,  it 
is  difficult  to  assert  that  pilots  adequately  recognized  the  lateral  situation 
discrepancies.  Thus,  detection  of  both  types  of  discrepancies  was  difficult. 

Contained  in  Appendix  Fisa  data  summary  from  the  MLS  Post  Mission 
Questionnaire.  In  essence,  those  pilots  who  initially  accepted  the  DFDS 
continued  to  do  so,  and  those  who  first  distrusted  it  maintained  their 
skepticism.  Regarding  the  MLS  End  of  Study  Questionnaire,  pilot  comments 
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TABLE  3 


ANALYSIS  OF  VARIANCE  SUMMARY  TABLE 
FINAL  APPROACH  TRACKING  PERFORMANCES 


Source 

Sum  of  Squares 

df 

Profile  Type  (A) 

240,014.95 

7 

Subjects  (S) 

293,012.36 

13 

Error  Term 

530,666.78 

91 

(A&S) 


Mean  Square  F Ratio 

34,287.85  **5.88 

22,539.41 
5,831.50 


**  5.88;  p .01  F (7,91) 
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TABLE  4 


FINAL  APPROACH  PERFORMANCES 


NEWMAN-KEULS 

ANALYSIS 

SUMMARY 

TABLE 

PROFILE 

3 

4 5 

7 

1 

8 

6 

2 

PROFILE 

3 

18.5  43.3 

63.4 

91.4 

*108.2 

*125.0 

*132.8 

4 

24.8 

44.9 

72.9 

89.7 

* 106.5 

* 114.3 

5 

20.1 

48.1 

64.9 

81.7 

89.5 

7 

28.0 

44.8 

61.6 

69.4 

1 

16.8 

33.6 

41.4 

8 16.8  24.6 

6 7.80 

2 

* p .01 
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TABLE  5 


DISCREPANCY  DETECTION 


Failure  Type  Percent  Detection 


I 

68 

1 1 A 

93 

I IB 

74 

IIIA 

4 

1 1 IB 

0 
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correlated  highly  with  pilot  tracking  performance,  in  that  those  profiles 
cited  as  hard  to  fly  were  so.  Some  criticism  of  the  testing  procedure  was 
given.  The  lack  of  a more  dynamic,  complicated  approach  environment  in  which 
other  aircraft,  air-to-ground  cormunications , and  intra-crew  coordination 
are  typical,  may  have  caused  discrepancy  detection  percentages  to  be  too 
high  and  unrepresentative  of  an  actual  situation. 


SECTION  IV 


DISCUSSION 

The  crossleg  performance  data  displayed  in  Figure  7 suggests  that  the 
degree  of  heading  change  required  to  intercept  the  crossleg  course  could  have 
affected  crossleg  performance.  In  essence,  the  greater  the  course  change 
required,  the  more  the  opportunity  for  error  existed.  On  the  profiles  flown, 
the  amount  of  course  change  required  to  intercept  the  crossleg  ranged  from 
0°  to  45°.  Profiles  2 and  4,  those  with  the  highest  mean  error  scores,  each 
required  40°  of  course  change.  This  was  at  least  15°  greater  than  that 
required  for  all  other  profiles,  except  for  the  45°  course  change  found  on 
profile  £.  However,  the  mean  combination  error  score  for  profile  8 was 
significantly  less  than  that  for  2 or  4.  An  examination  of  crossleg  lengths 
should  clarify  the  issue.  The  crossleg  lengths  of  profiles  2,  4,  and  8,  were 
1.7  nm,  2.8nm,  and  3.8  nm,  respectively,  and  suggest  that  the  longer  crossleg 
of  profile  8 helped  reduce  its  crossleg  mean  error  score. 

Two  factors  seem  to  be  interacting:  course  change  required  to  intercept 

the  crossleg  and  crossleg  length.  On  profiles  with  shorter  crosslegs  (i.e., 
less  than  3.0  nm),  the  amount  of  course  change  required  to  adopt  the  crossleg 
affected  tracking  performance.  On  such  segments,  significantly  higher  mean 
combination  scores  occurred  on  segments  with  high  intercept  angles.  Larger 
turns  were  more  difficult  and  such  difficulties  caused  tracking  errors  that 
were  too  great  to  be  quickly  corrected  on  short  segments.  However,  when 
operating  independently,  the  effects  of  a large  intercept  angle  (e.g., 
profile  8)  could  be  dampened  by  a long  crossleg  segment  and  the  influence  of 
a short  crossleg  could  be  offset  by  a shallow  intercept  angle  (e.g.,  profile 
1). 
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The  differences  among  scores  for  final  approach  performances  across 
profiles  shown  in  Figure  8 can  be  largely  explained  by  examining  the  various 
levels  of  intercept  distance.  For  example,  the  intercept  distance  of  profile 
3 was  4 nm,  while  those  of  profiles  2,  6,  and  8 were  about  2 nm.  The  arrangement 
of  mean  combination  scores  was  the  same  as  the  above  arrangement  of  intercept 
distances,  with  the  score  for  profile  3 being  significantly  less  than  those  of 
the  other  profiles.  Other  performance  differences  were  similar  and  indicate 
that  longer  final  approaches  fostered  superior  tracking  performance.  On  the 
longer  final  approaches,  pilots  spent  proportionately  less  of  the  total  tracking 
time  available  for  that  segment  recovering  from  the  intercept  turn  and  were  able 
to  devote  more  time  to  precisely  tracking  the  nominal  path. 

The  effect  of  intercept  angle  can  be  examined  by  comparing  the  mean 
combination  error  scores  for  profiles  with  equal  intercept  distances  but 
unequal  intercept  angles.  Comparing  the  mean  error  score  of  profile  3 to 
that  of  profile  4 and  the  mean  score  of  profile  2 to  that  of  profile  1 shows 
no  significant  differences  based  on  the  Newman-Keuls  test.  Varying  glide  slope 
angle  also  did  not  significantly  affect  the  mean  combination  scores. 

As  shown  in  Table  5 a type  IIA  discrepancy,  a nonparallel  displacement 
of  the  crossleg,  had  the  best  detection  rate.  The  HSI  clearly  displayed  the 
heading  error  and  the  course  error  and  continued  to  do  so  as  long  as  the 
aircraft  remained  on  the  crossleg.  The  presence  of  such  obvious  and  long 
duration  cues  enabled  the  pilots  to  consistently  detect  a type  IIA. 

However,  other  discrepancies  did  not  present  such  lengthy  detection  cues. 

A type  I IB  could  be  detected  only  during  the  turn  onto  or  off  of  the  crossleg. 
During  such  periods  pilots  needed  to  reference  the  DME  display  on  the  NCDU 
to  detect  the  error.  The  type  I IB  detection  rate  would  probably  have  been 
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higher  had  pilots  consistently  referenced  the  NCDl!  display  during  turns,  but 
they  occasionally  failed  to  do  this  due  to  pressure  to  satisfy  the  bank 
steering  bar. 

Discrepancy  types  IIIBX  and  1 1 1 BY  also  did  not  present  long  duration 
cues  and  could  be  discovered  only  during  the  turn  to  final  approach.  The 
nature  of  the  discrepancy  and  the  display  design  forced  pilots  to  use  a complex 
detection  procedure.  The  aircraft's  constant  descent  caused  the  glide  path 
error  values  on  the  NCDU  and  the  barometric  altimeter  readings  to  constantly 
change.  Further,  the  barometric  altimeter  had  50  feet  interval  markings  and 
an  error  tolerance  of  about  +_  50  feet  and  hampered  the  pilots  from  obtaining 
a useful  reading.  In  essence,  pilots  found  it  quite  difficult  to  check  the  NCDU 
and  the  alt  meter  quickly  and  accurately  and,  thus,  were  at  points  other  than 
the  checkpoint  when  obtaining  glide  path  error  and  altitude  values.  Finally, 
during  all  of  this,  the  pilot  was  pressured  to  satisfy  the  bank  steering  bar, 
which  had  been  leading  him  through  a turn  onto  the  final  approach  course.  The 
discrepancy  detection  procedure  was  quite  difficult  to  handle  even  given  ample 
time.  Shortness  of  time  forced  the  pilot  to  either  perform  the  computations 
hastily  and  inaccurately  or  to  ignore  them.  Thus,  the  discrepancy  went 
undetected. 

A type  1 1 1 AX  or  IIIAY  was  as  difficult  to  detect  as  either  type  1 1 1 B. 

When  tracking  a glide  path,  the  constant  throttle  adjustments,  pitch  changes, 
and  rolling  moments  caused  the  aircraft's  rate  of  descent  to  be  constantly 
changing.  The  rate  of  descent  oscillations  occurred  always,  regardless  of 
the  beam  type  of  glide  slope  angle.  The  changing  rate  of  descent  values 
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were  reflected  by  the  wide  range  of  VVI  pointer  oscillations,  which  made  it 
difficult  for  a pilot  to  accurately  compute  the  aircraft's  average  rate  of 
descent.  Thus,  a pilot  could  have  postulated  only  approximate  values  for  the 
aircraft's  rate  of  descent  (e.g.,  about  700-800  FPM),  which  made  it  extremely 
difficult  for  him  to  make  the  discrimination  that  the  aircraft's  true  rate 
of  descent  differed  by  100  FPM  from  that  expected.  Thus,  pilots  essentially 
could  not  detect  the  potentially  fatal  glide  path  angle  discrepancies  included 
in  this  study. 

A type  I discrepancy,  a nonparallel  displacement  of  the  final  approach 
course,  presented  long  duration  cues  as  did  a type  IIA.  The  25  percent 
difference  between  their  detection  rates,  although  not  statistically 
significant,  may  be  significant  from  a safety  of  flight  perspective.  A 
68  percent  detection  (type  I)  would  be  much  less  preferable  than  93  percent 
(type  IIA)  and  would  warrant  improvement,  especially  since  the  68  percent 
rate  was  for  a final  approach  discrepancy.  Further,  even  a small  navigation 
error  can  seriously  affect  flight  safety  on  the  final  approach  segment,  whereas 
a comparable  error  on  the  crossleg  would  be  uninfluencial . Thus,  a pilot's 
discrepancy  detection  should  become  more  acute  on  final  approach,  but  the  data 
suggests  that  this  may  not  be  true. 

The  pilot's  tasks  on  the  final  approach  were  similar  to  his  tasks  on  the 
crossleg,  in  that  he  was  to  keep  the  aircraft  under  control  at  all  times 
and  within  the  designated  airspace.  Traveling  from  initial  segments  to  and 
ever  the  crossleg  took  the  pilot  and  his  aircraft  through  an  air  corridor  of 
relatively  large  and  unchanging  dimensions.  Once  on  final  approach,  however, 
each  moment  of  flight  shrank  the  size  of  the  designated  airspace  and  caused 


the  pilot  to  devote  more  and  more  effort  to  keep  the  aircraft  within  said 
airspace.  As  the  pilot  devoted  more  attention  to  satisfying  the  command 
steering  bars,  he  could  give  less  to  crosscheck  items  like  the  HSI  course 
readout  and,  in  this  manner,  overlooked  the  type  I discrepancy. 

In  summary,  pilots  can  consistently  detect  a discrepancy  (i.e.,  type 
IIA)  if  the  needed  information  is  presented  for  a long  period  and  is  easily 
perceived  and  integrated  into  the  piloting  task.  However,  detection  becomes 
sporadic  when  the  relevant  i nformatior,  is  difficult  to  use,  pilot  workload  is 
high,  and/or  the  cues  are  of  a short  duration.  The  data  suggest  that  pilots 
need  position  information  that  is  easily  perceived  and  integrated  into  the 
piloting  task.  This  could  reduce  pilot  workload  by  simplifying  discrepancy 
detection  procedures  and  also  enable  pilots  to  improve  their  detection  of 
discrepancies  with  short  duration  cues.  However,  high  pilot  workload  can 
cause  even  obvious  cues  to  be  ignored,  which  suggests  that  providing  better 
position  i nformatior  will  not  completely  solve  the  problem. 

Another  approach  to  MLS  flight  safety  right  be  to  use  the  aircraft  and 
ground  station  to  reduce  the  p>  Utility  f a discrepancy.  Ground  station 
or  DFDS  alterations  could  insure  that  ,rly  an  accurate  and  safe  final  approach 
course  and  glide  s 1 opt  be  prograrr.,  : le.  In  (ffect,  such  a change  would 
establish  the  final  approai  ■ (.ootv.ir.iu  r tnr  PFl  S and  would  thereby  relieve 
the  pilot  of  programing  the  tina’  : * '.  The  role  of  the  ground  station 

and  DFDS  could  be  expanded  co  that  the  pt'.ot  or  the  air  traffic  controller 
could  select  a certain  approach  profile  and  input  all  the  profile  coordinates 
in  one  step.  The  pilot  then  would  be  unafle  t.o  make  errors  in  waypoint 
definition  and  selection  and  he  still  could  verify'  the  accuracy  of  the  coordinates 
prior  to  flying  the  approach.  However,  to  gain  user  acceptance  and  to  facilitate 
discrepancy  detection  when  the  above  cited  precautions  fail,  the  pilot's  ability 

J 

to  monitor  aircraft  position  during  an  approach  must  still  be  improved. 

I 
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SECTION  V 


CONCLUSIONS 

Intercept  angle  and  crossleg  length  interacted  to  produce  poorer  crossleg 
performance  on  profiles  with  short  crosslegs  and  high  intercept  angles. 

Leg  length  was  the  influential  factor  in  final  approach  performance. 

Longer  segments  allowed  pilots  to  more  effectively  recover  from  tracking  errors 
that  occurred  during  hte  previous  turn. 

Discrepancy  detection  was  problematic.  Detection  difficulties  point  to 
a need  to  provide  pilots  with  better,  not  necessarily  more,  position  information. 
Further,  a need  may  exist  to  limit  the  flexibility  of  the  MLS.  The  data 
indicated  that  MLS  flexibility,  while  being  one  of  its  more  potent  strengths, 
may  also  turn  into  one  of  its  most  serious  weaknesses. 
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APPENDIX  A 


PARAMETERS  RECORDED  EVERY  0.2  SECONDS 


Patern  Number 
Mission  Number 
Subject  Number 
File  Count 

Geographic  Altitude 

Latitude 

Longitude 

Position  on  A-axis  (Feet) 

Position  on  B-axis  (Feet) 

Lateral  error  from  nominal  track  (Feet) 
Vertical  error  from  nominal  track  (Feet) 
N updates  per  leg 

Indicated  Airspeed  (IAS)  (KTS) 

IAS  - 140  KTS 

Pi tch  Angle 
Roll  Angle 

Sin  of  Aircraft  Heading 
Cosin  of  Aircraft  Heading 
Throttle  position 
Rate  of  Cl imb 
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APPENDIX  B 


DETECTION  PROCEDDRES 

The  design  of  a type  I (figure  4)  was  affected  by  the  DFDS  design, 
such  that  both  the  initial  segment  and  the  final  approach  segment  had  to  be 
displaced.  However,  detection  of  the  inaccurate  final  approach  course  was 
the  real  item  of  interest.  A pilot  could  detect  the  inaccurate  final 
approach  course  by  either  referencing  the  course  readout  on  the  HSI, 
which  was  automatically  updated  by  the  DFDS,  or  by  looking  at  the  AAI , 

The  HSI  course  readout  would  have  read  176  instead  of  171  and  the  AAI 
would  have  its  pointer  to  the  left  of  the  0°  index,  which  represented  the 
final  approach  course,  instead  of  centered  over  the  0°  index.  In  either 
case,  pilots  were  told  to  cite  any  course  error  in  excess  of  + 2°  as  a 
discrepancy. 

A pilot  could  detect  a type  IIA  (Figure  5)  by  referencing  the  course 
readout  on  the  HSI.  Again  any  error  in  excess  of  + 2°  was  to  be  considered 
a discrepancy.  Since  no  winds  were  involved,  aircraft  heading  should 
coincide  with  the  intended  course  and  allowed  pilots  to  use  the  heading 
indicator  to  detect  a type  IIA. 

To  detect  a type  1 1 B (Figure  5)  required  the  pilot  to  cross  check  his 
DME  (to  the  station)  at  the  onset  of  the  turn  onto  the  crossleg.  If  this 
was  not  done,  the  discrepancy  could  still  be  discovered  if  the  pilot  checked 
the  DME  while  making  the  turn  onto  the  final  approach  course.  Discrepancies 
involved  errors  which  were  usually  about  .7nm  and  were  best  detected  using 
the  DME  readout  on  the  NCDU  data  display,  which  operated  during  every  approach 
and  pilots  were  told  to  reference  it. 

To  discover  either  a type  1 1 1 AX  or  III AY  (Figure  6)  the  pilot  needed  to 
estimate  the  aircraft's  average  rate  of  descent  by  observing  Vertical  Velocity 


Indicator  (VVI)  values  and  then  had  to  compare  his  rate  of  descent  estimate 
with  that  expected  for  that  glide  slope  value,  A difference  in  excess  of 
about  100  FPM  was  to  be  cited  as  a discrepancy* 

Either  type  1 1 1 B (Figure  6)  involved  offsetting  the  altitude  of  the 
computed  track  at  the  onset  of  the  turn  onto  final  about  100  feet  low 
IIIBX)  or  100  feet  high  ( 1 1 1 BY ) . The  discrepancy  detection  procedure 
required  the  pilot  to  reference  his  barometric  altimeter  first  as  the 
bank  steering  bar  displaced  laterally,  initiating  the  turn  onto  final* 
Timing  was  important.  After  checking  aircraft  barometric  altitude,  the 
pilot  had  to  immediately  note  the  glide  slope  error  on  the  NCDU.  If  he 
was  high  he  had  to  subtract  the  glide  slope  error  from  the  barometric 
altitude  value  and,  if  low,  he  had  to  add.  This  computational  procedure 
gave  the  pilot  the  altitude  of  the  computed  path  at  the  checkpoint,  a 
difference  between  the  computed  path  altitude  and  that  of  the  nominal 
one  in  excess  of  75  feet  constituted  a discrepancy. 
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PROFILE  DESIGN 
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appendix  d 

MLS  QUESTIONNAIRES 
MLS  END  OF  STUDY  QUESTIONNAIRE 


1.  Do  you  feel  that  the  dynamics  simulation  environment  encountered  in  this 
experiment  permits  you  to  make  a valid  judgement  regarding  the  feasibility 
of  making  instrument  approaches  in  the  MLS  environment? 

2.  Given  accurate  information,  which  profile  did  you  find  the  most  difficult 
to  fly?  Why? 

3.  Given  accurate  information,  which  profile  did  you  find  the  easiest  to  fly? 
Why? 

4.  What  potential  problems  do  you  see  in  using  the  present  simulator 
configuration  to  fly  the  given  profiles? 


MLS  POST  MISSION  QUESTIONNAIRE 


Name: 

Date: 

Missi on:_ 
Profiles: 


Yes 

No 

1. 

Did  you  have  any  difficulty  in  capturing  the  desired  course? 
Comment. 

Yes 

No 

2. 

Did  you  have  any  difficulty  in  capturing  the  desired  glide 
slope?  Comment. 

Yes 

No 

3. 

Did  you  have  any  difficulty  in  maintaining  your  horizontal 
orientation  with  reference  to  the  runway?  Comment. 

Yes 

No 

4. 

Did  you  encounter  any  difficulty  in  tracking  the  computed 
course  using  the  bank  steering  bar  or  the  CDI?  Comment. 

Yes 

No 

5. 

Did  you  encounter  any  difficulty  in  tracking  the  computed 
glide  slope  using  the  pitch  steering  bar  or  the  G/S  indicator? 
Comment. 

Yes 

No 

6. 

At  any  time  did  you  detect  that  the  computed  course  did  not 
accurately  coincide  with  the  prescribed  course  as  shown  on 

the  approach  plate?  Comment.) 
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7a.  If  the  answer  to  Question  #6  was  "yes",  then  on  which  profile 
and  by  what  means  did  you  detect  the  discrepancy? 

Yes  No  8.  At  any  time  did  you  detect  that  the  computed  glide  slope  did 

not  accurately  coincide  with  the  prescribed  glide  slope  as 
shown  on  the  approach  plate?  Comment. 

9a..  If  the  answer  to  Question  #8  was  "yes",  then  on  which  profile 
and  by  what  means  did  you  detect  the  discrepancy? 

9b.  Could  you  take  any  corrective  action? 

10.  Given  the  present  system  and  displays,  rate  how  confident 
you  would  be  with  regard  to  executing  a safe  approach  and 
landing. 

Very  unsure  Very  confident 

Comment. 

/ ‘ L 

• 11.  Do  you  have  any  recommended  improvements  to  the  present 

system  and  display? 
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APPENDIX  E 


Lateral  and  Vertical  AAE  Scores  Per  Profile 


1 Standard 

1 Standard 

Lateral  AAE 

Deviation  Unit; 

Vertical 

Deviation  Unit; 

Profile  Numbers 

Seqment 

Scores 

Lateral  Scores 

AAE  Scores 

Vertical  Scores 

1 

Crossleg 

183.5 

93.0 

37  5 

30.1 

Final 

192.3 

167  9 

54.6 

36.4 

2 

Crossleg 

362.6 

278.8 

52.0 

32.1 

Final 

216.5 

199.6 

54.9 

38.9 

3 

Crossleg 

162.0 

94.6 

44,7 

30.1 

Final 

88.5 

67.6 

47  5 

31.3 

4 

Crossleg 

306  2 

233.8 

59.0 

52.7 

Final 

121.4 

93.4 

48.4 

26.5 

5 

Crossleg 

90.0 

65.8 

69  4 

65.2 

Final 

111.9 

90.8 

73.6 

63.5 

6 

Crossleg 

103.2 

79.3 

59.0 

34.2 

Final 

213.5 

163.4 

55.6 

36.9 

7 

Crossleg 

151.2 

86.4 

70.9 

71.9 

Final 

144  1 

138.1 

76  6 

94.1 

8 

Crossleg 

168.6 

117.4 

60.9 

74.1 

Final 

163.5 

151.4 

92.2 

88  5 
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| APPENDIX  F 

POST  MISSION  QUESTIONNAIRE 
CONFIDENCE  RATING  RESULTS 


Subject 

SI 

S2 

S3 

S4 

S5 

S6 

S7 

Average 

4.1 

8.1 

4.5 

8.4 

7.6 

8.3 

4 0 

Subject 

S8 

S9 

S10 

SI  1 

SI  2 

SI  3 

SI  4 

Average 

5.6 

1.3 

4.5 

8.9 

4 . 5 

4.9 

8.8 

Mi ssion 

I 

II 

III 

IV 

V 

VI 

VII 

VIII 

AVG 

Average 

5.3 

5.5 

5.9 

6.2 

6.5 

6.7 

6.4 

6.5 

6.0 

